Add DeepSeek V4 GB300 Dynamo TensorRT-LLM agentic config / 添加 DeepSeek V4 GB300 Dynamo TensorRT-LLM Agentic 配置#2274
Conversation
Add eight disaggregated MTP topologies and the agentic routing, container-cache fallback, and GB300 launch support they require. 中文：添加八组分离式 MTP 拓扑，并补齐所需的 Agentic 会话路由、容器缓存回退及 GB300 启动支持。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Replace the placeholder with the published upstream PR URL. 中文：将性能变更日志中的占位链接更新为已发布的上游 PR URL。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I didn't find any bugs, but this PR touches shared infrastructure (benchmark_lib.sh's dynamo session-routing logic and the GB300 launcher's squash-import locking/fallback path) in addition to adding eight new agentic recipes, so it's worth a human look before merging.
What was reviewed: the new session-control branch in build_replay_cmd (X-Session-ID vs the legacy conv-aware-routing flags), the refactored squash-import lock/fallback flow in launch_gb300-nv.sh, the new dynamo-trt agentic dispatch branch, and the eight new recipe YAMLs plus the nvidia-master.yaml/perf-changelog.yaml entries.
Extended reasoning...
Overview
This PR adds eight new disaggregated MTP prefill/decode topology recipes for DeepSeek V4 Pro FP4 on GB300 using Dynamo TensorRT-LLM, plus supporting changes: a new session-routing branch in benchmarks/benchmark_lib.sh (opt-in X-Session-ID header vs legacy AIPerf conv-aware-routing flags), a new dynamo-trt/dsv4 dispatch branch in runners/launch_gb300-nv.sh that clones srt-slurm v1.0.30, a reworked squash-file import path (configurable primary dir + local-fallback with a proper write-probe and atomic tmp-then-rename), master-config and perf-changelog entries.
Security risks
No injection, auth, or credential-handling concerns. The squash-import rework uses srun ... bash -s -- with positional args (avoiding the prior string-interpolation-into-heredoc pattern), which is a modest security/robustness improvement. All values driving the launcher come from repo-controlled recipe config, not external user input.
Level of scrutiny
This PR mixes low-risk additive changes (new YAML recipes, master-config, changelog) with a shared-infrastructure change: the squash-import/lock rework and the new benchmark_lib.sh session-routing branch affect every GB300 job and every dynamo-* recipe respectively, not just the eight new configs. That combination — new recipes are easy to sanity-check, but the launcher/library changes are shared blast-radius — pushes this past the bar for a mechanical/config-only approval, even with zero reported bugs.
Other factors
The bug-hunting system found no bugs, and the four candidate issues raised by finders (batch_sizes vs max_batch_size, squash-lock move to /tmp, squash fallback dir not being shared storage, and the pyxis '#'-to-'/' image name handling, plus the literal-string 'None' env var) were all examined and ruled out this run. The recipe still carries the standard CODEOWNER sign-off requirement per the bot's reminder comment, which has not yet been completed in the visible timeline.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29657037702 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
1 participant
Summary
batch_1,benchmark, andsa-sharedcluster values.中文说明
batch_1、benchmark和sa-shared配置。